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Exploring the 2018-2019 i-Ready Predictive Capability 


Curriculum Associates’ i-Ready is an adaptive diagnostic and individualized instructional tool that 
has been used in M-DCPS in the last few years. In addition, Curriculum Associates provides the 
District with results of their predictive model, which uses the students’ outcomes on the Fall and 
Winter i-Ready diagnostic testing as well as the schoolwide percentages of students scoring 
proficient on the previous year’s Florida Standards Assessments (FSA) in English Language Arts 
(ELA) and Mathematics to estimate the individual student probabilities scoring at every 
achievement level on the subsequent FSA ELA and Mathematics grades 3-8. 


Predictive Capability of i-Ready was first examined in June 2019 and described in the 
corresponding Research Brief (http://drs.dadeschools.net/ResearchBriefs/RB 1608.pdf). This Research 
Brief examines 2018-2019 i-Ready prediction data and their relationship with the 2018-2019 FSA outcomes 
in ELA and Mathematics. 


To enable a comparison of the i-Ready predictive results with the 2019 FSA observed results, the 
following procedure was used. The 2018-2019 Winter i-Ready’s reported probabilities of scoring 
in each of the achievement levels 3-5 on the 2019 FSA were added, and the results were converted 
into a dichotomous variable coded as 1 if the probability of scoring within achievement levels 3-5 
was at least 0.5, and O otherwise. In addition, achievement level on the 2019 FSA was 
dichotomized and coded as | if a student scored within achievement levels 3-5, and 0 if the student 
scored within achievement levels 1-2. 


The results of the comparison of the 2019 FSA ELA and the predicted Winter i-Ready results for 
Grade 3 students, as an example, are shown below. 


2019 FSA ELA 


2018-2019 Winter i-Ready Levels 1-2 | Levels 3-5 | Total 
Predicted to score in Levels 1-2 6211 1051 7262 
Predicted to score in Levels 3-5 2077 10591 12668 
Total 8288 11642 19930 
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It can be seen that of the 7262 Grade 3 students who were predicted to score within achievement 
levels 1-2 on the 2019 FSA ELA, 6211 in fact scored that way. The corresponding cell in the table 
above is generally referred to as containing the True Negative (TN) results. Similarly, of the 12668 
students who were predicted to score within achievement levels 3-5, 10591 in fact scored that way. 
These are generally referred to as True Positive (TP) results. 


One measure of predictive success is its Accuracy, defined as a percentage of correct predictions. 


Correct Predictions _ TN+TP 6211+10591 
For the table above, Accuracy= ———————— = = ———— _ = 84% 
Total Total 19930 


Because predicted and actual achievement levels may agree by chance, the accuracy results are 
generally accompanied by a statistical measure of agreement that corrects for a chance agreement. 
One such popular measure is Cohen’s Kappa. It achieves a maximum value of | if the predicted 
and actual results are identical. Values of the Kappa in the 0.61 — 0.80 range are often interpreted 
as indicators of a substantial agreement, while the values in the 0.41 — 0.60 range as indicators of 
a moderate agreement. 


Below are the results of an analysis of the capability of the Winter i-Ready diagnostic results to 
predict the 2019 FSA ELA and Mathematics results by grade level and subject area. 


ELA Mathematics 
Grade | Accuracy | Kappa | Accuracy | Kappa 
I 84% 0.67 83% 0.61 
4 83% 0.64 85% 0.62 
5 84% 0.66 84% 0.62 
6 83% 0.66 85% 0.69 
7 83% 0.66 82% 0.64 
8 83% 0.66 78% 0.52 
Total 84% 0.66 83% 0.64 


The values of prediction accuracy and the Kappa coefficient reported in this table are very similar 
to those reported in June 2017. 


It can be observed that the results displayed in the table above demonstrate a high accuracy of the 
Winter 1-Ready results for predicting the 2019 FSA outcomes in both subjects and across all grade 
levels shown. In addition, they show a substantial degree of agreement (corrected for the chance 
agreement) between the predicted and actual outcomes in both subjects and across all grade levels, 
except for Grade 8 in mathematics. In that one case, the value of the Cohen’s Kappa falls within a 
moderate agreement range. 


It should be noted that accurately predicting FSA achievement levels in grade 8 Mathematics is 
challenging because a considerable portion of the grade 8 students take Algebra 1 course and 
participate in the corresponding EOC assessment instead of the FSA Mathematics. 


Additional Measures of Predictive Success 


Predictive abilities of various tests are often reported using several additional measures. In the 
remainder of this brief, some of these measures are defined and presented. 


Looking back at the table on the first page, one can see that 1051 students in Grade 3 were predicted 
to score within achievement levels 1-2 but scored instead within achievement levels 3-5 on the 
2019 FSA ELA. These cases are referred to as False Negatives (FN). Similarly, 2077 students 
were predicted to score within achievement levels 3-5, but instead scored within achievement 
levels 1-2. These are referred to as False Positives (FP). 


These are four additional measures of a predictive success of a test: 


att . ' : TP : 
e Sensitivity is the percentage equivalent of the following fraction a In our context it 


answers the following question: of all students who scored within achievement levels 3-5, what 


percent were predicted to score that way? 

e Specificity = a is the answer to the following question: of all students who scored within 
achievement levels 1-2, what percent were predicted to score that way? 

e Positive Predictive Value (PPV) = — answers the question: of all students who were 
predicted to score within achievement levels 3-5, what percent scored that way? 

e Negative Predictive Value (NPV) = — answers the question: of all students who were 


predicted to score within achievement levels 1-2, what percent scored that way? 


The table below presents these additional indices. 


ELA Mathematics 
Grade Sensitivity | Specificity | PPV | NPV_ | Sensitivity | Specificity | PPV | NPV 
6 91% 75% 84% | 86% 93% 64% 84% | 82% 
4 88% 76% 86% | 79% 95% 61% 85% | 85% 
5 91% 74% 83% | 85% 94% 65% 83% | 86% 
6 88% 78% 83% | 84% 92% 76% 83% | 88% 
ZL 89% 77% 82% | 85% 91% 73% 77% | 89% 
8 90% 74% 83% | 84% 72% 81% 65% | 85% 
Total 89% 75% 84% | 84% 92% 70% 82% | 86% 


It should be noted that the values of the Sensitivity shown in the table above are somewhat higher 
than those found in 2017, especially in mathematics. On the other hand, the values of Specificity 
based on the 2019 data and exhibited in the table above are somewhat lower than the corresponding 
values found in 2017. That reflects an inverse relationship between Sensitivity and Specificity of 
any predictive measure. 


Summary 

The results of the analyses presented above indicate that various indices of a test’s predictive 
success are sufficiently high with a possible exception of the values of the Cohen’s Kappa and the 
Positive Predictive Validity for Grade 8 students in mathematics. Therefore, the usage of the 
i-Ready diagnostic assessment as a predictive tool for the FSA outcomes in grades 3-8 is justified. 


